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ABSTRACT 

The Student Information System <SIS) is designed to 
provide a longitudinal data base about students matriculating at and 
graduating from the College of Education at Ohio State University. 
The SIS has four basic purposes: (1) to document student experience 
for accountability and accreditation purposes; (2) to diagnose 
student progress for general student advising and counseling 
functions; (3) to collect data about students and programs for 
evaluation of both graduates and programs; and (4) to research the 
nature of teacher education and teacher ^eveloi^ent and other 
professional personnel programs. This report contains a description 
of the SIS. A matrix display illustrates how information is obtained 
on each student at three stages of their careers: (1) 
preprofessional; (2> preservice professional; and (3) inservice 
professional. A summary report of the 1982-83 results of the National 
Teacher Examinations administered to graduating seniors is included, 
as well as the Executive Summary Follow-up Survey of Teacher 
Education Graduates 1978-79, 1980-82, and 1981-82. (JD) 
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Program Evaluation at The Ohio State University 
William E. Loadman 



Rationale 

Within the last three years, the College of Education at The Ohio 
State Itolverslty has been worMng on the creation and In^lementatlon of 
a system for documenting and assessing the ej^erfences and abilities of 
all teacher candidates, toward the Improvement of Its teacher education 
programs. Initially this system was created because of dissatisfaction 
with what has becom the accepted means of program evaluation In teacher 
education; that Is, throu^ foUw-up studies. The general criticism of 
this single vehicle for program evaluation is that the results of follow-up 
studies do not provide ccmcluslve information on which to base continued 
development and Improvement of teacher education programs. Further, their 
summatlve ndt\ire alone does not provide the kind of formative Information 
necessary for an effective and responsive teacher education program. 

One view of program tfevelopment and inprovemeift requires that 
developers 'sngige In the ongoing process of "developmental Inquiry" 
(Sanders, 1981 which provlte for the generation of hypotheses through 
a data collection process that Informs further development of programs. 

Developmental Inquiry requires a docimentation and assessment system 
that can provide a rich contextual accounting of both teacher candidates 
and programs. The intent Is first to <tetcrm1ne what Is happening in the 
programs, concurrent with <teterroin1ng v<iat should happen. With this 
perspective In mind the assessment and docwnentatlon system was designed 
and titled the Student Information System (SIS). 

At this time, efforts are focused on Implementation at the program 
area level and in those courses (or experiences) that are comB©n for all 
teacher candidates. The aim Is to provide an overall picture of the 
teacher education program for the tollege. Instrimentatlwi has been 
developed, piloted and refined for several of these general professional 
experiences, and In several of the program areas. 



Purposes 

The SIS has four basic purposes. They are: 

1. to document student experience for accouriwabillty and 
accreditation purposes; 

2. to diagnose student progress in programs in order to fulfill 
general student advising and counseling functions; 
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3. ' to collect data about our students and programs for purposes 

of evaluation of both graduates and prograns, toward program 
Improvements; and 

4. to research the nature of teacher education and teacher 
development and other professional personnel programs. 

Length of Time In Use 

Program evaluation activities in the College of Education at The 
Ohio State University have a substantial history which can be traced to the 
pioneering efforts of Ralph Tyler In what has been classically termed the 
Eidit Year Stu<<y prominent in the late 19^'s and 1940's. The work of 'Guba 
in the mid sixties and Stufflebeam In the late sixties and early seventies 
adds to these prominent activities. With such a legacy, one could anticipate 
continued leadership In regard to program evaluation. This tradition of 
innovation and leadership continues with the development and Implementation 
of SIS. 

Fomal follow-up studies of graduates of the preservice education 
program were Initiated on an annual basis In 1976 as one aspect of program 
evaluation. In 1980 the SIS was Initially conceptualized and expanded the 
efforts In program evaluation. This system has continued to (tevelop and 
grw until the present. 

Approach 

The SIS 1s designed to provide a longitudinal data base 
students matriculating through and graduating from the College of Education 
at The Ohio State University. The system is designed to be nwltlj 
dimensional. Including the collection of data at nwltlple points In the 
professional education program, using multiple data collection vehicles 
to triangulate the perspectives of teacher candidates and campus and field 
teacher educators 1nvol\«d in the College's teacher education programs. 

The context for the system reflects the College's and profession's 
need to compile more adequate descriptive information about programs 
candidates, and to use that data to effectively evaluate the quality oK 
our candidates and the progran6 they complete, as well as to improve the\ 
on-going program development process. 

The theoretical framework for the evolution of this system Is two- fold: 

(a) the need to Improve teacher evaluation by linking knowledge of teacher 
candidates' abilities to perform In teaching roles (Medley, 1977); and 

(b) the need to develop systems for the evaluation of teacher education 
programs that go beyond previous suramative follow-up studies to | more 
formative process of data-based program development (Sanders, 1981 J. 
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As can be seen on the matrix display* Information Is obtained on each 
student at three general stages In their careers: (a) preprofesslonal ; 
(b) preservlce professional; and (c) Inservlce professional. These stages 
are presented on the horizontal dimension of the matrix. The vertical 
dimension of the matrix represents the following types of Information: 
(a) descriptive; (b) assessment; (c) narrative; and (d) contextual. The 
cells within the matrix identify the specific data which is collected- 
As can be readily Inferred, the system is large and complex. Substantial 
resources are necessary to operate this system; at times the logistics 
behind the system are staggering. The system includes observational, 
perfor..iance, assessment and survey data, thus getting multiple measures 
and perspectives on each student. The system continues to be developed. 
Implemented and refined. f 



Sources of Data 

As indicated on the matrix, there are many and varied sources o^ 
being used in the system. These Include high school records, facult 
assessments, standardized exam performance, student self-ass«»ssment, 
supervisor assessment, cooperating teacher assessment, and university 
doc«nents. The data sources are designed to be multl faceted and 
triangulated where possible. 



HATRIX CF THE 
STUDENT INFWWATIOH SYSTEM 

College of Education 
The ©ilo State University 



VS4 



PERSONAL DATA ^ 


EMROLLHENT OATA 2 


DE6REE DATA 3 


1* SSN 6. Eteploywit 
Z. Full name 7. Sex 
3- Current address B. Marital 

4, Cltirenship status 

5, Itote of birth 9. Race 

10. Physical 
Impalfpnent 


1, Year of ^st 5. Itejor 

recent admission 6. No. of qtrs* en*- 
2r Qkwrter of most rolled In ptxtgrm 

recent adtelsslw 7. Total hn. tailed 

3. airrent enroll- at O.S.U. 

ment status B. Hrs. earned In a 

4. College given quarter 


1 . I^te ex^wcted 
2« P^}*ee 

3. Oeoree rece1vi«l 

4. College 

5. Date received 



ERLC 



TEACHER EDUCATION Pmilt PROGRESSION 
DATA CacmNTS 



DE5W3PTOK 

{factual descrip* 
tors of d^rt- 
nmU enrailmentt 
courses* experl- 
et^est decisions) 



AS&ES9ffiWT 

[J) psj^loalcal 

dtaract^stlcs 



knoirledgn 
skills 
beliefs 



in, HA»«ATnf£ 

(fmiltlplt per- 
spective 
connentary and 
analysis of 
ecperlemes) 



IV. CONTEXT 

^descriptions of 
envlronmnt use- 
ful In Inter- 
preting experiences 



A) PRE- 

PIRSFESSIONAL 


High retool nam 
High school addrtts 

Graduatirm class size 
Cla^s ramc 
% rank 


High sctiool SPA (2) * 
ACT 12) 
SAT (2) 

* 


D 


7 


B} PRESERVICE 
WOFBSTOTAL 

General Education 

Elective Courses 

Fr^hman Early 
Experiencing 
Proqram 

Professional 
Introduction 

Special ^^ods 

Foundations 

Content Specialty 
Courses 

Student Teaching 
Otr» of Graduation 


o 

Course Infomatlon 


A 

toirse grades (2) 


1ft 
i V 


11 

0 if 


Coi^rse Information 


Course grades (2) 


14 


15 


Course Informatum 
Credit hwrs 
Oai casipus 
Field piacei^t 
B1o1nvent;>ry • 
End of charter 
questionnaire 


-WTI (1) " 
PRF {!) 

Course gnides (2} 
Exploration 
profile (TCP) (3) 
End-qusrter eval. (5} 


Personal groirthpl^.i^^ 
Critical event fo»w 


19 


Cwrse information ^ 
Credit hours 
OSO cmpus 
Field placement 


Cwrse grade (2) 21 
leacner canoioaw 
profile (TCP) (3) 
Connwn exm (2) 


Critical event form22 


23 


24 

Course 1nfonr»t1on 


2S 

Course graifes (2) 
TCP^{3) 


Critical event fonn^^ 


27 


28 

Course Infomatlon 


29 

Cmirse graites (2) 


30 


31 


32 

Course Information 


33 

Cwrse grades (2) 


34 


35 


36 

CouVse infcmation 
Field placement 


TCP* ^ 37 
ffi>servation scale 
Supervisor reoMJ. (5) 
C«irse grade (5) 


38 

Critical event form* 


39 


40 


NTE (2) 


42 


43 


€• INSERVICE 
WSmSTIONAL 


44 

Fo11<^^p dem^ 
graphic survey 


5RE (2) ^ 
Folloi^-yp demo- 

grafAlc suty^y (5) 
Foil e«*-up observation 
Follow-up supervisor 

survey 

8 


^ 46 
FollOM-up interview 

Follow-up supervisor 

Interview 

i 


47 

1 



^nder developi^nt 



October, 1983 
THE OHIO STATE UNIVERSITY - 

C0LLB5E OF EDUCATION 

* r 

Summary Report of the 1982-83 Results 

of the National Teacher Examinations * . * 

Introduction 

This summary reports the results from the secom!* annual adnlnlstratlon of 
the National Teacher Examinations (NTE) with groups of graduating seniors from 
The Ohio State University College of Education. The 1981-82 test was conAicted 
with Autwnn Quarter graduates and the 1982-83 test with Spring Quarter graduates. 
This report presents the results from the total 1982-83 test group. 

In November 1982 the four NTE common Examinations: professional education; 
English; social studies, literature, and fine arts;^ and science and mathematics 
(3 hours, 15 minutes total test time) were replaced by the Core Battery. The're- 
fore comparisons between the two years of performance assessme.nt are not straight- 
forward. The Core Battery consists of three tests: Professional Knowledge, 
General Knowledge, and Cawnuni cation Skills, each of which Is tw hours long. 
The 1982-83 administration covered only the Professional Knwledge test and 
the Specialty Area tests. The Professional Knowledge test Is designed to deter- 
mine the student's knowledge of teaching skills and practices (pedagogy). In 
addition, students take the specialty area exam In their respective areas of 
preparation, e.g., Emjllsh eAicatlon majors take the English Specialty exam, 
mathematics education majors take the mathenatlcs exam, etc. 

1982-83 Administration and Results 

Sample » 

Over 500 Spring Quarter graduating seniors representing all College of 
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Education teacher education programs were informed of the opportunity to take 
the NTE and that a sample would be selected from respondents. Fran the 162 
on) who responded 100 were randomly selected from the program areas for which 
Specialty Area tests are available. After cancellations the nimiber of students 
who completed the tests was 79 (15%) representing eight program areas. Due 
to Incomplete data on some variables, the n«nber was 76 for the SpedaHy Area 
tests and 56 for ACT scores. Table 1 presents the frecpaency of students taking 
the exam from eight program areas. 
Demociraphic and Performance Variables 

Table 2 presents selected Information on the demographic and performance 
measures. Students age 21-25 ccaiprlsed 9U of the NTE sample. Those whose 
ages were 26-35 equaled 8% and only one student (1%) was over age 35. Almost 
B0% were female. The sample Included students from eight program areas Including 
Elementary, English, Exceptional Children, Home Economics, Hathematlcs, Music. 
Physical Education and Social Studies. The largest proportion (34.2%) of the 
sample Is from elementary education. The mean GPA for the group Is 3.21 . Com- 
pared to the 1981-82 NTE seniors the GPA's from this group Is slightly higher. 

ACT scores for this group are also somewhat higher than the 1981-82 NTE 
group. This year's mean score (22.59) Increased by 1.2 points (the national 
average Is 20.0). The breakdown of this year's scores, compared to nationally 
standardized groups of elementary and secondary education. Is well above the 
national average: SB% of the students scored above the national average; only 
7% of the students fell Into the lowest quartlle on the ACT.^ 
NTE Professional Knowledge Scores 

The mean scaled sore of the 1982-83 group of seniors on the Professional 
Knowledge exam Is 666 which according to the national norms ranks at the 72nd 
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percentile and 10 score points above the national wean of 656. The standard 
deviation of the scores Is 8.4. Table 3 shows the dlstrltatlon of scores by 
percentile rank and scaled scores for the average of all students within 
each program area. Rie to revision Qf the NTE conroon examinations the tests 
differ for the 1981-82 and 1982-83 years and therefore are not canparable. 
NTE Specialty Area Exanlnatlon Scores 

The NTE specialty area exam was administered to students from eight 
selected teacher education programs and the averaged performance of students 
In each program area Is presented In table 3. This exam provides a measure 
of the s-^dents* competence In their major area of concentration. I.e., their 
specialty area. The specially area exams did not change from the 1981-82 
administration and therefore the results from the 1981-82 sample are directly 
comparable to the 1982-83 ssnple. The scores are reported In two ways; 
(a) a scaled score and (b) a national percentile rank. The scaled scores 
cannot be compared across progr«n areas because each specialty area has Its 
own normative (reference) distribution. Therefore an overall group average 
has not been computed on this measuJ^. However, the scaled scores and the 
respective percentile ranks have been generated for each separate program area. 
Based upon the national norms of the National Teachers Exam, students In the 

i 

eight College of Education programs scored between the 60S11e and the 87S11e 
on their respective specialty area exans. The results from the 1982-83 
administration of this portion of the NTE are similar to the results of the 
1981-82 administration, ftese results are very positive and encouraging. 
Correlation of Demographic Variables and NTE Scores 

A Pearson proluct moment correlation coefficient using the scaled scores 
Showed an extrsnely high positive relationship (p. <.001) among the variables 
of GPA, ACT, and NTE (scaled scores). Table 4 presents the correlation matrix 

11 
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for the slx'dasographlc and performance vari^Bles. Age and sex showed no 
relationship to measures of performance. The correlation among the 6PA, 
ACT and NTE scor^ Indicates a substantial amount of conreon variance among 
the measures; I.e., there Is a positive relationship between entrance score 
performance (ACT), schoo) performam:e (St»A) and outcome performance (NTE). 

Analysis of Variance * ' , 

Elqht one-way analyses of variance comparisons were made. The four 
dependent variables Included the two NTE scores, 6PA, and ACT scor? . Each 
was analyzed by the Independent variables of program area and sex. Only one 
statistically significant difference was found and It showed that students in 
on4 program area scored higher on the Professional Knowledge test than students 
In one other program area. All other differences were found to be not signlf- 
Icantly different. 



^ Amerlan College Testing Program. Assessing students on the wky to co;neqe^- 
College stud ent profiles: Norms for the A^T assessment iVol. z?. lowa City, 
Iowa: ACT Publications, Wi. 
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Table 1 



OSU College of Education 
Frequency and Percent of Graduates taking the 
National Teacher Examinations Spring (garter 1983 



Progran 


rrOTeSS lOna i 

Knowvii^e Test 


Proar^Area 


Si^cl&l tv 
Area Test 


% by 
Program Arei 


1. Elementary 


27 


34.2 


26 


34.2 


2. Exceptional Children 


11 


13.9 


11 


14.5 


3. English 


11 


13.9 


10 


13.2 


4. Music 


9 


11.4 


9 


11.8 


5. Physical Education 


8 


10.1 


8 


10.5 


6. Hathematici 


6 


7.6 


6 


7.9 


7. Social Studies 


4 


5.1 


3 


3.9 


8. Home Economics 


3 


3.8 


3 


3.9 




79 


100.0 


76 


99.9 



• 
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Table 2 

Selected Demographic Data on Students Taking the 
National Teachers Exam 1982-83 



Age 

21-25 

26-35 



% 



36 and older 1% 
Itean age » 23.48 
N - 79 



Sex 

Male 

Female 



H 

16 

79 



% 

20 

80 



Grade Point Average 
mean « 3.21 
N « 79 



Average ACT Score 
mean » 22.59 
N « 56 



. ACT national Averages 
Top Quartlle 76 - 100 Xlle 

Upper Middle Quartlle 51 - 75 «ile 
l<mr Middle (^artlle 26-30 %11e 
Bottom Quartlle 1 - 25 %11e 



% of Local Students 
Included in National 
worm categories 

m 

2S% 
25% 

7% 



100% 
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Table 3 



Perfarwance on Professional Knowledge and 
Specialty Area Tests of the National Teachers Exaai 1982-83 

Professional Knowledge Specialty Area 

"Scaled scaled 

H Prooraa Area Score ^ Scor« MS. 

27 1. Elenentary Education ^ 867 74 858 68 

11 2. English Education 668 76 637 71 

n 3. Exceptional Child Education 672 85 669 79 

3 4. Home Economics Education 669 81 C83 82 
6 5. Mathematics Education 662 62 665 84 
9 6. Music Education 658 50 627 60 
8 7. Physical E&icatlon 663 64 697 87 

4 8. Social Studies Education 664 67 610 62 

firoup Average 666 72 
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Table 4 



Correlation Analysis of NTS and 
N. Demographic and Performance Variables 1983 





AfiE 1 


SEX 


GPA 1 


ACT 


SPEC 


PROF 


, .095 


.0^ 


.491* 


.481* 


.614* 


SPEC 


-.117 


-.083 


.481* 


.484* 




ACT 


-.135 


-.149 . 


.542* 






GPA 


-.048 


.074 








SEX 


-.028 























*p. <.001 

The saaple sizes for the correlation coefficient range frtai a low of 56 
to a high of 79. 

ACT « American Col lege Test " » 

ASE « Age of TMcher 

6PA a Srade Point Average 
PROF « Professional Knowledge 
SPEC » Specialty Area Exam 

SEX n Sex of Teacher 
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EXECUTIVE SUPMARY 
FOLKSrf-UP SURVEY OF TEACHER EDUCATION 
GRADUATES 1978-1979, 1980-1981. and 1981-1982 
COllEGE OF EDUCATION 
THE OHIO STATE UNIVERSITY 

OVERVIEW 

The following is an executive suBnaiy of Technical Report #8 of the Follow- 
Stwfy of The Ohio State Itolverslty's Teacher Education Programs. The present 
study Is on graduates of the College of Education for the academic years 1978- 
1979, 1980-1981, and 1981-1982. This study Is one In a series of studies on 
the College of Education's graduates conducted since 1977. Hiese studies are 
conducted In part to meet the standards of the National Council for the Accredl- 
tat-'on of Teacher Education (NCATE) and the C»»1o State Department of Education's 
standards for evaluating. teacher education students. 

In the past years onlv a sample of first year teachers were surveyed for 
the follow-up study; this year In addition to all 1982 graduates, a 20 percent 
random sample, stratified by program area, of 1978-79 graduates an'd 
1981-1982 were surveyed. This method allows for more accurate comparisons - 
between sai^le years and elite for assessment, over time, of such factors as 
satisfaction with employment, tsefulness of educational preparation and feelings 
about the teaching profession. The sample sizes were as follows: 

1981-1982 Graduates 961 (entire population) 

19^1981 Graduates 193 

1978-1979 Graduates 213 
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The response rate for each year Is: 

1981-1982 597 62S 

190)- 1981 .113 Sn 

1978-1979 138 m 

In addition to the changes In the saBplIng procedure, changes were made In 
the data collection techniques. The qliestlcmnaire was studied arid changes In 

» . ' . * 

the wording of certain Iters were made, other Items were ellnrfnated and new 
items Included. The questionnaire was structured to obtain Information regarding: 
present job status; satisfaction with job; student teaching experience; att1tu<tes 
toward preservlce academic training; educational background and aspirations; and 
demographics. 

Statistical Analysis and Reporting 

In previous years the data collected from the follow-i^ qi^tlonnalre were 
analyzed primarily by conputlng frequencies and percentages for each Item. From 
that analysis a profile was developed of the sample and some comparisons made 
with the previous year. The analysis for this year was more extensive. 

• First a chl -square to determine the representativeness of the respondents 
by program area and sex for each sample year was performed. In addition, 
descriptive statistics Including means, standard deviations, frequencies, and 
percentages were produced for each Item. 

^ From these results a description or profile of the students was developed 
for each san?)le year. Con^arlsons between sample years were ma<te and differences 

examined using analysis of variance techniques. Comparisons were also made 

between thv» following groups within each year: 

IS 



(1) Program Areas 

(2) Teaching Level (elementaiy, middle, secondary) 

(3) Sex 

(4) Current Eoiploynent Subgroups 
Results 

The follaM-i4> questionnaire yielded ^ large annunt of Infomiatlcm about 
the graduates surveyed from the three sanple years. The 1980-1981 sasq»le and 
the 1981-1982 samples both proved to be representative of their populations 
on. both program area and sex. The 1978-1979 sample was representative of 
its population cm the sex variable but not on the program area variable." 
The nonrepresentatlveness on the program area variable was due to the over 
safflpttnrt>^small program areas In order to Include ie«0Op"¥uBJects*to ' '^^ 
produce stable statistical results for these program areas. The Impact 
on this situation on the outcome of the study was found to be negllble and 
therefore the results present a valid profile of graduate of the college. 
Analyses Indicated that there was very little difference among the sample 
years. In addition, the comparisons made between sex, among program areas 
(academic majors), enployment subgroups and teaching produced some interest- 
ing and important findings. Briefly, s<»ne of those findings are: 

1- The majority of the graduates (75?) are female; yet there has 
been a progressive increase in the nwitjer of males graduates 
from samplB year to sample year. 
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2. Over 90 percent of the graduates are en^loyed but approxloately 
1/3 are In noneducation related positions. 

3. Although the graduates are generally satisfied with their current 
positions, those teaching are significantly more satisfied than 
those In education related or noneJucatlon related employment. 

4. The majority of the students {n%) felt that personal Initiative 
was the awstja^rtant strategy for securing empldyment. 

5. Within the teaching employment subgroup, those Individuals 
teaching the longest were more satisfied with their jobs than 
the more recent teachers. 

6. The location of the graduates' current teaching positions can be 
grouped Into the following community types: 

Urban t%% 
Suburban 35% 
Rural 41% 

7. Fifty-five percent of the teachers are teaching at the senior 
high level; 27 percent are teaching at the elenentary level and 
18 percent at the junior high llvel. 

8. Sixty-six percent of the teachers feel that supervision of 
extracurricular activities Is voluntary and 55 percent of the 
teachers actually supervisor extracurricular activities. 

9. Generally, the graduates reported their student teaching 
experience to be quite suc<»ssful. For example, 98% of the 
graduates rated their experience as somewhat successful or 
successful; 88 percent reported having a good or very 90od 
relationship with their cooperating teacher. 
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10. Seventy- five percent of the students completed all four years 
at The Ohio State University. 

11. Approximately 50 percent expressed a desire to obtain an 
advanced degree in education; another "25 percent plan to 
obtain one In a noneducati on field. 

Because the samples, primarily, were representative of their populations, 
these' findings can be generalized with confidence to the target populations 
of College of Education graduates or specific progranj areas* The complete 
Technical Report of the follow-up process and findings can be obtained from 
Hiniam Loa<toan at (614). 422-1257. In addition. Individual program area 
results can also be requested. 
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